Add the ability to convert from/to awkward array by AreWeDreaming · Pull Request #108 · iterorganization/IMAS-Python

AreWeDreaming · 2026-03-17T16:31:57Z

The tests (imas/test/test_wrangle.py) that I have designed are passing now.
The wrangler.py adds two functions, wrangle and unwrangle.

The wrangle method takes the flat representation of the data and wrangles into the nested IDS.
The unwrangle method takes a dictionary of ids and uses the IDSTensorizer to flatten out a defined list of entries.

- Wrangling works - Unwrangling awkward data needs work

AreWeDreaming · 2026-03-18T12:55:16Z

I am holding off on running black to keep the diff clean

imas/backends/netcdf/ids_tensorizer.py

imas/wrangler.py

Handle version propagation more cleanly Co-authored-by: Prasad <prasad.sawantdesai@gmail.com>

Remove preference for specific version Co-authored-by: Prasad <prasad.sawantdesai@gmail.com>

…-Python into awkward_array_support

…is provided in unwrangle

imas/wrangler.py

AreWeDreaming · 2026-04-01T14:41:51Z

Note that if IMAS-Python is installed with

pip install --no-deps --no-build-isolation -e .

The imas/_version.py is not created causing imas to fail on import.
Adding imas/_version.py with

version="0.0.0"
version_tuple=[0,0,0]

manually fixes this but is clearly not the desired solution. How can we get pip install to work properly?

prasad-sawantdesai · 2026-04-01T17:22:37Z

Note that if IMAS-Python is installed with
pip install --no-deps --no-build-isolation -e .
The imas/_version.py is not created causing imas to fail on import. Adding imas/_version.py with
version="0.0.0"
version_tuple=[0,0,0]
manually fixes this but is clearly not the desired solution. How can we get pip install to work properly?

Could you point out setuptools and setuptools_scm version in your environment when you are using --no-build-isolation flag?
I quicky tested without no-build-isolation flag and it works. pip install --no-deps -e .

AreWeDreaming · 2026-04-01T18:26:23Z

To reproduce you can try these instructions:
https://github.com/GA-FDP/imas_composer/tree/imas_python?tab=readme-ov-file#temporary-setup-to-export-to-standard-ids-assumes-omega-cluster
Note that you would not do it on the DIII-D omega cluster so you skip the module load conda step. It will ultimately fail because you lack access to the MDSplus server but the import error should fire before that. You can of course also just open a python session and import imas to trigger the error.

prasad-sawantdesai · 2026-04-03T08:27:18Z

Tried locally on SDCC

(myenv)$pip install wheel setuptools_scm # ensure you have wheel and setuptools_scm in installed in the environment

(myenv)$ pip list                                                                                                                                                       
Package        Version     Editable project location                                                                                                                                                           
-------------- ----------- --------------------------------------                                                                                                                                                                                                                                                                                         
packaging      26.0                                                                                                                                                                                            
pip            23.2.1                                                                                                                                                                                          
setuptools     65.5.0                                                                                                                                                                                          
setuptools-scm 10.0.5                                                                                                                                                                                          
vcs-versioning 1.1.1                                                                                                                                                                                           
wheel          0.46.3                                                                                                                                                                                                                                                                                                                                                                                                  
[notice] A new release of pip is available: 23.2.1 -> 26.0.1                                                                                                                                                   
[notice] To update, run: pip install --upgrade pip 

(myenv) [sawantp1@98dci4-srv-1005 IMAS-Python]$ pip install --no-deps --no-build-isolation -e .                                                                                                                Obtaining file:///home/ITER/sawantp1/github/IMAS-Python                                                                                                                                                        
  Checking if build backend supports build_editable ... done                                                                                                                                                   
  Preparing editable metadata (pyproject.toml) ... done                                                                                                                                                        
Building wheels for collected packages: imas-python                                                                                                                                                            
  Building editable for imas-python (pyproject.toml) ... done                                                                                                                                                  
  Created wheel for imas-python: filename=imas_python-2.2.1.dev21-0.editable-py3-none-any.whl size=10092 sha256=fd5c022cc1e89b8ed09e038b348b0333be3f6410a67bca8d3e5668f36ca4c09d                               │
  Stored in directory: /tmp/pip-ephem-wheel-cache-m2239qmz/wheels/a9/0d/81/e451d06713deb1130f483b94d6634714ddca7d73da76c729fb                                                                                  
Successfully built imas-python                                                                                                                                                                                 
Installing collected packages: imas-python                                                                                                                                                                     
Successfully installed imas-python-2.2.1.dev21                                                                                                                                                                 
[notice] A new release of pip is available: 23.2.1 -> 26.0.1                                                                                                                                                   
[notice] To update, run: pip install --upgrade pip

… wrangler.py

Add wrangler

4228cad

- Wrangling works - Unwrangling awkward data needs work

AreWeDreaming requested review from maarten-ic, olivhoenen and prasad-sawantdesai as code owners March 17, 2026 16:31

AreWeDreaming added 2 commits March 18, 2026 10:06

Fix errors in test data and handle 2d ragged data

7faa192

Extend tests to strings

17ad3b8